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COCHLEAR IMPLANT SOUND PROCESSING METHOD AND SYSTEM 

The present invention relates generally to the generation of electrical stimuli 
for application to an auditory prosthesis electrode array forming part of a cochlear 
5 implant. 

Cochlear implants are now the clinical device of choice for people who have 
a profound hearing loss. Such implants typically consist of an internal and an 
external component. The internal component consists of an array of electrodes 
implanted within the cochlea, and associated electronics for providing current 

10 stimulation to the surviving auditory neural elements. The external device is a 
sound processor, which converts an incoming acoustic signal into appropriate 
electrical stimulation pulses for application to the electrode array. 

Cochlear implant technology has progressed to the stage that the average 
user can expect to understand conversational speech in quiet surroundings without 

15 the aid of lip reading. However, cochlear implants do not provide a satisfactory 
hearing experience when listening to speech in the presence of other noises, and 
when listening to musical sbunds and in particular when listening to melody. Inhere 
are thought to be two principle reasons for this: firstly, the pitch of complex 
sounds, generally related to the fundamental frequency, is poorly conveyed by 

20 current speech processing schemes, and secondly, the fine temporal and spectral 
information that normally hearing people use to segregate a complex sound from 
noise or other complex sounds is lost or reduced in implant processing. 

Fine temporal information in acoustic sounds is represented by relatively fast 
amplitude fluctuations in the wave form of the signal. For periodic (or harmonic) 

25 complex signals, such as voiced speech or musical sounds, the fundamental 
frequency (F0) is related to the period of fluctuation, and leads to the perception of 
musical pitch or voice pitch. The fundamental frequency of these sounds generally 
lies between 50 Hz and at least several hundred Hz. Finer temporal information is 
provided by the frequencies of the harmonic components of such sounds, which are 

30 at multiples of the fundamental frequency. 
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The signal processing applied in current cochlear implant sound processors 
does not allow the fine frequency perception that this would require, either 
spectrally or temporally. In the spectral dimension, up to about twenty fixed narrow 
band filters are usually assigned tonotopically to the electrode positions in the 

5 cochlea. This arrangement does not allow the implantee to resolve harmonic 
components of complex sounds, and hence to perceive the fundamental frequency 
via the spectral template method. In the temporal domain, the output of the filters 
will have amplitude modulations at the fundamental frequency rate, provided that 
the filters are wide enough to encompass more that one harmonic, and provided that 

10 smoothing functions at the output of the filters do not attenuate this modulation. In 
theory, these modulations could provide a fundamental frequency pitch perception, 
provided that the rate of stimulation is high enough to accurately "sample" this 
modulation pattern. 

However, recent studies have shown that users of one of the most common 

15 processing strategies used in cochlear implants, namely the Spectral Peak or 
SPEAK strategy, could not discriminate the pitch of some complex sounds. 
Inspection of Sie processor output showed amplitude modulations for low 
fundamental frequencies, but the modulations on different electrodes appear to have 
somewhat random phase relationships. It has also been shown that phase shifts on 

20 nearby electrodes will impair the ability of the implantee to perceive the modulation 
frequency, or cause them to perceive a pitch not related to the fundamental 
frequency. 

It would therefore be desirable to provide a cochlear implant sound 
processing strategy to improve the perception of fundamental frequency for speech 
25 and musical stimuli. 

It would also be desirable to provide a cochlear implant sound processing 
strategy to allow the important components of a complex sound, such as a vowel 
sound or a musical note, to be perceptually grouped by an implantee. 

It would also be desirable to provide a cochlear implant sound processing 
30 strategy to allow two or more concurrent complex sounds, such as two speakers, to 
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be better perceptually segregated by an implantee. 

It would also be desirable to provide a cochlear implant sound processing 
strategy that ameliorates or overcomes one or more disadvantages of the prior art. 

With this in mind, one aspect of the present invention provides an auditory 
5 prosthesis device for selectively stimulating electrodes within an auditory prosthesis 
electrode array, comprising: 

(a) a transducer for converting a complex acoustic sound into an electrical 

signal, 

(b) signal processing means responsive to an electrical signal and 
10 generating a temporal pattern of stimulation pulses to selected electrodes within the 

electrode array, the stimulation pulses being applied to each electrode at an 
electrode stimulation rate, 

(c) feature extraction means for deriving an estimate of at least one 
fundamental frequency of the electrical signal, and 

15 (d) stimulation pulse adjustment means for adjusting the stimulation 

pulses in accordance with the estimated fundamental frequency. 

Ir? a preferred embodiment of the invention, the future extraction means 
comprises a plurality of fundamental frequency templates separating the electrical 
signal into a plurality of different frequency components, each template comprising 

20 a first template filter centred on a first frequency and one or more further template 
filters centred on harmonics of that first frequency, wherein 

the stimulation pulse adjustment means acts to compare output signals from 
the template filters in each template to determine a matching template passing 
maximum power compared to the remaining templates, the first frequency of the 

25 matching template being used by the stimulation pulse adjustment means as the 
estimated fundamental frequency. 

The stimulation pulse adjustment means may selectively generate stimulation 
pulses during an electric stimulation period, the pulse adjustment means acting to 
convert the estimated fundamental frequency into the electric stimulation period 

30 using a specific conversion function. 
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In one embodiment, the specific conversion function may perform a first 
function of setting the longest interval between consecutive pulses in the temporal 
pattern during the electric stimulation period in accordance with the estimated 
fundamental frequency. 
5 Additionally, or alternatively, the specific conversion function may perform 

a second function of setting the electric stimulation period in accordance with the 
estimated fundamental frequency. 

The specific conversion function may vary between performing the first and 
second function. 

10 The specific conversion function may preferentially perform the second 

function when the electrodes to be stimulated in the temporal pattern are physically 
proximate each other. 

In the presence of two or more complex acoustic sounds, the feature 
extraction means may derive an estimate of multiple fundamental frequencies from 

15 the electrical signal and the specific conversion function may convert the multiple 

fundamental frequencies into a corresponding number of interleaved electric 

S " /& 

stimulation periods. * 

In another embodiment of the invention, the stimulation pulse adjustment 

means may modulate the amplitude of the stimulation pulses applied to an electrode 

20 within the electrode array which corresponds to the estimated fundamental 

frequency. 

The stimulation pulse adjustment means may further act to process the output 
signals from the template filters in the matching template to determine an estimate 
of one or more formants of the electrical signal. 
25 The stimulation pulse adjustment means may modulate the amplitude of the 

stimulation pulses applied to one or more electrodes within the electrode array 
which correspond to the estimated one or more formants. 

When the feature extraction means derives an estimate of multiple 
fundamental frequencies, each corresponding to a different complex acoustic sound, 
30 from the electrical signal, the stimulation pulse adjustment means may temporally 
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segregate the amplitude modulation applied to the stimulation pulses for each 
different complex acoustic sound. 

In a further embodiment, the stimulation pulse adjustment means may act to 
apply a temporal offset to the stimulation pulses applied to those electrodes within 
5 the electrode array which correspond to the estimated fundamental frequency or 
formants of the electrical signal with respect to the stimulation pulses applied to 
other electrodes in the electrode array. 

The stimulation pulses may be applied simultaneously to those electrodes 
within the electrode array which correspond to the estimated fundamental frequency 

10 or formants of the electrical signal. 

Alternatively, the stimulation pulses applied to those electrodes within the 
electrode array which correspond to the estimated fundamental frequency or 
formants of the electrical signal may be temporally segregated from the stimulation 
pulses applied to the other electrodes in the electrode array. 

15 Another aspect of the invention provides a bilateral auditory prosthesis 

apparatus including two auditory prosthesis devices as described hereabove. 

The stimulation pulse adjustment iheans of a first of the auditory prosthesis 
devices may act to determine a matching template passing maximum power 
compared to the remaining templates in the first auditory prosthesis device, the first 

20 auditory prosthesis device further acting to determine the power passed by the 
corresponding template in the second auditory prosthesis device and, if less than the 
power passed by the matching template in the first auditory prosthesis device, 
adjusting the stimulation pulses in the first auditory prosthesis device in accordance 
with the estimated fundamental frequency. 

25 In the presence of two complex acoustic sounds, the stimulation pulse 

adjustment means of the first auditory prosthesis device may act to determine to 
matching templates passing power maxima compared to the remaining templates in 
the first auditory prosthesis device, the stimulation pulse adjustment means of the 
first auditory prosthesis device adjusting the stimulation pulses in the first auditory 

30 prosthesis device in accordance with the estimated fundamental frequency 
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corresponding to a first power maximum, and the stimulation pulse adjustment 
means of the second auditory prosthesis device adjusting the stimulation pulses in 
the second auditory prosthesis device in accordance with the estimated fundamental 
frequency corresponding to the second power maximum. 
5 The stimulation pulse adjustment means of the first auditory prosthesis 

device may act to apply stimulation pulses to one or more electrodes in the first 
auditory prosthesis device which correspond to one or more formants of the 
complex acoustic sounds, and the stimulation pulse adjustment means of the second 
auditory prosthesis device may act to apply stimulation pulses, which are de- 

10 correlated with the stimulation pulses applied by the first auditory prosthesis device, 
to one or more electrodes in the second auditory prosthesis device which 
corresponds to the same one or more formants. 

A further aspect of the invention provides a method of operating an auditory 
prosthesis device for selectively stimulating electrodes within an auditory prothesis 

15 electrode array, the method including: 

(a) converting a complex acoustic sound into an electrical signal, 

(b) responsive to the eJectrical signal, generating a temporal pattern of 
stimulation pulses to selected electrodes within the electrode array, the stimulation 
pulses being applied to each electrode at an electrode stimulation rate, 

20 (c) deriving an estimate of at least one fundamental frequency of the 

electrical signal, and 

(d) adjusting the stimulation pulses in accordance with the estimated 
fundamental frequency. 

The following description refers in more detail to the various features of the 

25 invention. To facilitate an understanding of the invention, reference is made in the 
description to the accompanying drawings where the auditory prosthesis device and 
method of its operation are illustrated in a preferred embodiment. It is to be 
understood however, that the invention is not limited to the preferred embodiment 
as shown in the drawings. 

30 In the drawings: 
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Figure 1 is a schematic diagram showing various components of the sound 
processor of one embodiment of an auditory prosthesis device according to the 
present invention; 

Figure 2 is a graphical representation of the pass bands of the first four 
5 template filters within each of three fundamental frequency templates forming part 
of the sound processor of Figure 1, and a harmonic input spectrum to be processed 
using these templates; 

Figure 3 is a graphic representation of a temporal pattern of electrical 
stimulation pulses applied to an electrode array in an auditory prosthesis device; 

10 Figure 4 is a flow chart showing schematically the series of functional 

operations performed by the sound processor of Figure 1, and 

Figure 5 is a schematic diagram of a part of bilateral auditory prosthesis 
apparatus including two sound processors. 

Existing auditory prosthesis devices include a microphone that converts 

15 sounds into electrical signals, and a sound processor responsive to the electrical 
signal from the microphone and selectively generating stimulation pulses for 
application to electrodes within an electrode array. In multi-cfiannel cochlear 
implants, the electrode array is inserted in the cochlea so that different auditory 
nerve fibres can be simulated at different places in the cochlea. Typically, the 

20 electrode array includes 22 electrodes. Different electrodes in the electrode array 
are stimulated by the pulses from the sound processor depending upon the 
frequency of the electrical signal received from the microphone. The electrodes 
near the base of the cochlea are stimulated with electrical pulses derived from high 
frequency acoustic information, while electrodes near the apex are stimulated with 

25 electrical pulses derived from low frequency acoustic information. The sound 
processor acts to break the input signal into different frequency bands and 
selectively apply stimulation pulses to different electrodes within the electrode array 
based upon the signal in each of the frequency bands. Typically, the microphone 
and speech processor are worn by the patient, and stimulation pulses from the sound 

30 processor are transmitted to the electrode array by a transmitter/receiver pair. 
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Figure 1 shows a more detailed view of the various components of the sound 
processor. The sound processor 10 implements, in this embodiment, a Continuous 
Interleaved Sampling (CIS) sound processing strategy. The electrical signal from 
the microphone 2 is firstly pre-emphasised and then passed through a bank 1 1 of 
5 fixed overlapping bandpass filters. In this example, there are 22 bandpass filters 

(BFP1, BFP2 BPF22), corresponding to the 22 electrodes within the electrode 

array 4. The envelopes of the filtered waveforms are then extracted by a bank 12 of 
full wave rectifiers and low-pass filters. The envelope outputs are then compressed 
and used by a pulse generator 13 to modulate biphasic pulses applied to the 

10 electrodes (El-1, El-2, El-22) in the electrode array 4. In order to avoid channel 

interaction, trains of the biphasic pulses are delivered to the electrodes in a non- 
overlapping or non-simultaneous fashion, i.e., in a way such that only one electrode 
is stimulated at a time. The amplitude of the pulses is related to the amplitude of 
the envelopes detected by the envelope detectors. 

15 Whilst the sound processing strategy used in this embodiment of the 

invention conforms to the CIS approach, it will be appreciated that other strategies 
mat involve the passing of the electrical signal through a bank of fixed overlapping 
bandpass filters, the outputs of which determine the amplitude of the pulse applied 
to the corresponding electrode, may also be adopted. Examples of such strategies 

20 include the Advanced Combination Encoders (ACE) strategy, and the Spectral Peak 
(SPEAK) strategy. Alternatively, the amplitude of the stimulation pulses applied to 
each electrode may be determined from an acoustic loudness model together with 
an electrical loudness model, such as in the Specific Loudness (SpeL) strategy. 

The sound processor 10 further comprises a bank 14 of fundamental 

25 frequency templates for deriving an estimate of at least one fundamental frequency 
of the electrical signal from the microphone 2. The bank 14 of fundamental 
frequency templates separates the electrical signal from the microphone 2 into 
different frequency components. Each fundamental frequency template comprises a 
first template filter centred on a first frequency within the frequency component and 

30 one or more further template filters centred on harmonics of that first frequency. In 
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the example depicted in Figure 2, the fundamental frequency template FOi includes 
a first template filter centred in practice on a fundamental frequency F0 l5 and 3 
template filters centred respectively on harmonics of that frequency, namely 2F0|, 
3F0! and 4F0|. In practice though, a fundamental frequency template may include 
any number of filters, and may typically include ten filters. 

Similarly, the fundamental frequency templates F0 2 , F0 3 F0i 2 shown in 

Figure 1 each include a first template filter centred on a fundamental frequency and 
a series of filters centred respectively on harmonics of that frequency. 

The frequency spacing between the fundamental frequency on which the first 
template filter in each template is centred will depend upon the ability of the 
cochlea implantee to distinguish sounds. Typically, the values of the fundamental 
frequencies detected by the templates may be derived from the standard musical 
scale. In one embodiment, the fundamental frequencies may be spaced from each 
other by a semi-tone. 

Each fundamental frequency template defines an ordered list of frequencies. 
Each frequency specifies the lower or upper sound of a different fundamental 
frequency and its associated harmonics that may potentially be present in the signal 
detected by the microphone 2. The upper and lower sounds may be spaced by a half 
semi-tone on either side of the centre frequency for each harmonic. For example, 
the fundamental frequency template for the note "A" would define the upper and 
lower bounds of the first template filter at 213.7 Hz and 226.4 Hz (220 Hz ± 0.5 
semi-tone), and the upper and lower bounds of the subsequent template filters at 
427.5 Hz and 452.9 Hz, 641.2 Hz and 679.3 Hz, 854.9 Hz and 905.8 Hz, and so on. 

The output signals from the template filters within each fundamental 
frequency template filter are provided to a stimulation pulse adjustment device 15. 

Figure 2 illustrates the pass bands 20, 21 and 22 of the first four template 
filters in each of the first three fundamental frequency templates FOi, F0 2 , and F0 3 
shown in Figure 1 . Input signal power that is passed by the template filters with the 
template FOi for example, is assumed to arise from a harmonic signal with a 
fundamental frequency approximately equal to FOi, whereas the input power that is 
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not passed by the filters within this template is assumed to arise from noise. 

An example of a harmonic input spectrum 23 of a signal from the 
microphone 2 to be processed by a set of templates including template filters having 
the pass bands shown in Figure 2 is also depicted. The input spectrum has a 
5 fundamental frequency of F0 X . The lowest four harmonics of F0 X are shown, 
starting at F0 X . Although the magnitudes of the harmonic components vary, it can 
be seen from this figure that the largest amount of power contained in the spectrum 
would be passed by the template having filters centred on F0 2 and its harmonic 
components, when compared to the amount of power passed by the other templates. 

10 Accordingly, the fundamental frequency FO2 can be taken to be an estimate of the 
value of the fundamental frequency F0 X . 

The pulse adjustment device 15 compares the signals from the filters in each 
of the templates in the bank 14 to determine the template passing the maximum 
power compared to the other templates. The first frequency upon which the first 

15 filter within that template filter is centred is taken to be an estimate of the 
fundamental frequency of the signal from the microphone 2. 

Furthermore, the output signals of the filters within the template passing the 
maximum power can be processed by the pulse adjustment device 15 to extract 
local spectral peaks. In effect, the filters within that template act to sample the 

20 complex harmonic signal detected by the microphone 2, and further processing of 
this sampled signal can provide an estimate of the formant frequencies in the case of 
voiced speech or a similar type of sound. 

To better determine the template(s) passing the maximum power compared 
to the other templates, a weighting function may be applied by the pulse adjustment 

25 device 15 to the outputs, of all filters in the templates. For example, a weighting 
may be applied to increase the value of the output signal strength of those templates 
centred on a fundamental frequency in the mid-range of human hearing. For each 
template, the outputs of the filters are amplified or attenuated according to their 
centre frequencies. The range of centre frequencies over which an appropriate 

30 waiting function is applied may typically be between 500 to 1500 Hz. 
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It is to be understood that in other embodiments of the invention, other 
techniques can be used to provide a real-time estimate of the fundamental frequency 
of a complex harmonic signal detected by the microphone 2. 

An advantage of the above described technique for identifying the 
5 fundamental frequency of the electrical signal is that, as stated above, it also 
identifies the frequency components of the complex harmonic signal. 

A further advantage of this technique is that it enables the identification of 
two concurrent complex sounds. In this case, there with be maxima at the output of 
two fundamental frequency templates within the bank 14, each maxima 
10 corresponding to an estimate of a separate fundamental frequency. 

Furthermore, the outputs of the filters in these two templates will specify 
which spectral components belong to each fundamental frequency. 

A third advantage is that the signal to noise ratio can be estimated by 
comparing the total power at the output of the matching fundamental frequency 
15 template with the total power of those components of the signal that do not coincide 
with any of the filters in the matching fundamental frequency template. Such a 
signal to noise ratio estimate is an indication of the reliability of the estimated * 
fundamental frequency value(s). 

To that effect, the stimulation pulse adjustment device 15 additionally detects 
20 the level of the total overall signal captured by the microphone 2 without being 
firstly filtered by the bank 14 of templates. The total overall signal is a combination 
of the complex harmonic signals (if any) and noise that is detected by the 
microphone 2. The total overall signal strength may be compared by the pulse 
adjustment device 15 to the signal strength at the outputs of the matching 
25 fundamental frequency templates to determine either a complex harmonic signal-to- 
noise ratio or a complex harmonic signal-to-overall signal ratio. If one or both of 
these ratios is below a predetermined threshold value, no pulse adjustment may be 
applied. 

Alternatively, no pulse adjustment may be applied if the pulse adjustment 
30 device 15 determines that the overall signal strength is below a predetermined 
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threshold value and that the estimate of the fundamental frequency of the complex 
harmonic signal is likely to be unreliable. 

If none of the outputs of the templates are determined by the pulse 
adjustment device 15 are determined to have a power greater than any of the others 
5 (i.e. a high level signal is detected at all templates), no pulse adjustment may also be 
applied. 

Rather than simply applying no pulse adjustment in the above-described 
scenarios, the pulse adjustment device 15 may alternately set the stimulation rate of 
the pulses applied by the pulse generator 13 to either a fixed high rate of stimulation 

10 or a random stimulation rate. 

The estimations of the fundamental frequency (or fundamental frequencies) 
can be used by the pulse adjustment device 15 to modify the "usual" output of the 
pulse generator 13 in a variety of ways. Each of these adjustments allows the main 
components of the complex sound to be perceptually grouped by the cochlear 

15 implantee, and also allows the implantee to better perceive the fundamental 
frequency pitch. 

In a first technique, the stimulation pulse adjustment means applies 
information derived from the estimated fundamental frequency to the temporal 
pattern of pulses generated by the pulse generator 13. Figure 3 shows one example 

20 of such a temporal patter of pulses applied to electrodes within an electrode array of 
a cochlea implant. In this example, pulses are delivered to the electrodes in a non- 
overlapping fashion so that only one electrode is stimulated at a time. Each 
electrode is stimulated at an electrode stimulation period 30 determined by the 
interval between consecutive pulses applied to a same electrode. The temporal 

25 pattern of pulses shown in Figure 3 is applied to selected electrodes within an 
electrode array of cochlea implant during an electric stimulation period, or control 
period, referenced 31. The longest interval between consecutive pulses in the 
temporal pattern shown in Figure 3 is referenced 32. 

In its simplest form, the . stimulation pulse adjustment means can set the 

30 electrode stimulation rate to be equal to the estimated fundamental frequency. 
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However, psycho-physical research has indicated that, where multiple pulses are 
applied to an electrode in an electrode stimulation period, the pitch perceived is 
related to the longest interval in the pattern, rather than the stimulation period. 
Furthermore, it has been shown that stimulation on nearby electrodes is perceptually 
5 amalgamated to form an overall pitch perception. Accordingly, the specific 
conversion function used by the pulse adjustment device 15, in a first embodiment, 
sets the longest interval between consecutive pulses in the temporal pattern during 
the electric stimulation period in accordance with the estimated fundamental 
frequency. In this way, an appropriate pitch verses fundamental frequency 
10 conversion is produced for implantees. The principle is applicable when multiple 
pulses are present in the temporal electrical stimulation pattern for each electric 
stimulation period. 

For example, it is possible to run through each group of active electrodes in a 
usual ACE or CIS scheme more than once within each electric stimulation period, at 

15 least for speech sources or other low values of fundamental frequency. With 
existing, fast cochlea stimulators, eight selected electrodes may typically be 
activated on each of, for example, four stimulation cyclfes with a time interval that is 
shorter than half the period of a typical speech fundamental frequency. Thus, even 
if stimulation gaps of length equal to the period of the fundamental frequency are 

20 incorporated into the electric stimulation period, an overall high rate of stimulation 
can be achieved. 

The specific conversion function may also be used to set the electric 
stimulation rate at which pulses are applied to individual electrodes in accordance 
with the estimated fundamental frequency. 

25 The specific conversion function may also vary between setting the longest 

inter-pulse interval to correspond with the estimated fundamental frequency, and 
setting the electrode stimulation rate to correspond with the estimated fundamental 
frequency. The specific conversion function may vary between the two different 
conversions applied to the stimulation pulses in a gradual manner. The specific 

30 conversion function may preferentially perform the setting of the longest inter-pulse 
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interval to correspond to the estimated fundamental frequency particularly where 
the electrodes to be stimulated in the temporal pattern are physically proximate to 
each other. 

Where the bank 14 of templates derives an estimate of multiple fundamental 
5 frequencies from the electrical signal, thereby indicating the presence of multiple 
complex sounds, the specific conversion function converts the multiple fundamental 
frequencies into a corresponding number of interleaved electric stimulation periods. 
It may be that only one or two electrode stimulation cycles are present for each 
electric stimulation period. 

10 Another technique that may be implemented by use of the pulse adjustment 

device 15 involves the imposition of an in-phase fundamental frequency amplitude 
modulation to the output of the pulse generator 13. In its simplest form, the 
amplitude of every activated electrode can be modulated by the pulse adjustment 
device 15 in accordance with the estimated fundamental frequency. 

15 A more sophisticated strategy may be to modulate the amplitude of the 

stimulation pulses applied to one or more electrodes within the electrode array 
which correspond to the estimated one or more formants. As described previously, 
the output signals of the template filters in the matching fundamental frequency 
template can be processed to determine an estimate of one or more formants of the 

20 complex acoustic sound. 

The concurrent and in-phase amplitude modulation of the stimulation pulses 
applied to the formant electrodes improves perception of the fundamental frequency 
pitch of the complex sound. Moreover, the "marking" of the formants with an 
amplitude modulation corresponding to the fundamental frequency of the complex 

25 sound allows these important components to be perceptually grouped as belonging 
together, and segregated from any concurrent sounds or noise. 

Keeping the modulation pattern smooth as the fundamental frequency 
changes can be achieved, for example, by using a modulation weighting function in 
which the phase of the function varies smoothly. That is, the phase of the weighting 

30 for. the next pulse can be an increment of the previous phase, the size of the 
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increment being determined by the next fundamental frequency estimate and the 
time elapsed. 

The above-described techniques can be used in the case of two or more 
concurrent complex sounds. Where the bank 14 of templates derives an estimate of 
two fundamental frequencies, each corresponding to a different complex acoustic 
sound, there will be two templates that provide significant output signals, or power 
maxima. In this case, there will be two fundamental frequencies each with different 
formant estimates provided by the output of two matching templates within the 
bank 14 of fundamental frequency templates. Thus, there may be four electrodes 
which correspond to the estimated formants, which are marked with appropriate 
amplitude modulation patterns. The different modulation rates on the two different 
pairs of electrodes will enable the formants to be paired to make correct vowels and 
for the two vowels, or musical sounds, to be heard as perceptually separate. 

An alternative arrangement involves the stimulation pulse adjustment device 
applying a temporal segregation of the amplitude modulation applied to the 
stimulation pulses for each different complex acoustic sound. For example, a 
temporal offset may bl applied to the stimulation pulses applied to those electrodes 
within the electrode array which correspond to the estimated fundamental frequency 
or formants of the electrical signal, with respect to the simulation pulses applied to 
other electrodes in the electrode array. The stimulation pulses may be applied 
simultaneously to those electrodes within the electrode array which correspond to 
the estimated fundamental frequency or formants of the electrical signal. 

Alternatively, the stimulation pulses applied to those electrodes within the 
electrode array which correspond to the estimated fundamental frequency or 
formants may be temporally segregated from the stimulation pulses applied to other 
electrodes in the electrode array. For example, the group of electrodes 
corresponding to components (the fundamental frequency and/or formants) of the 
complex acoustic signal may be activated in close succession, following by a 
temporal gap before activation of any additional electrodes. 

As seen in Figure 4, each of the above described variants all have a common 
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initial processing step in which an acoustic signal 40 is processed in a conventional 
manner, in a first step 41, to determine an electrical level to be applied to each of 
the electrodes in the electrode array 4. In concurrent step 42, the fundamental 
frequency of the acoustic signal 40 is extracted. Preferably, the formants of the 
complex harmonics signal are also extracted together with the harmonic 
components of the signal. At step 43, the information extracted at step 32 is used to 
adjust the simulation pulses conventionally produced by existing sound processing 
techniques. 

The above described techniques can also be adapted to an bilateral auditory 
prosthesis apparatus in which a separate auditory prosthesis device is provided for 
each ear of a user. Figure 5 illustrates one example of such a bilateral apparatus. A 
first microphone 50 converts acoustic signals into an electrical signal provided to 
the input of a bank 51 of bandpass filters and envelope detectors, and also to a bank 
52 of spectral template filters. A pulse generator 53 selectively generates 
stimulation pulses to electrodes within a first electrode array implanted in a first ear 
of a user. Information relating to the fundamental frequency of the electrical signal 
is derived from the output of the filters in the bank 52 by £ pulse adjustment device 
54. The pulse adjustment device 54 acts to modify the stimulation pulses generated 
by the pulse generator 53 in the manner described in relation to Figure 3. 

Similarly, a second microphone 60 is provided to convert an acoustic signal 
into an electrical signal provided to the input of a bank 61 of bandpass filters and 
envelope detectors and to a bank 62 of spectral template filters. A pulse generator 
63 acts to selectively generate stimulation pulses to electrodes within a second 
electrode array implanted in the other each of a user. Information relating to the 
fundamental frequency of the electric signal, provided by the bank 62 of template 
filters, is used by a pulse adjustment device 64 to modify the stimulation pulses 
generated by the pulse generator 63. In its simplest form, the auditory prosthesis 
devices operate independently and separately process complex sounds detected by 
their respective microphones. However, in the example shown in Figure 5, 
processing carried out in one auditory prosthesis device impacts the processing 
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carried out in the other auditory prosthesis device. 

The above described information relating to the fundamental frequency of 
the complex signal detected by the microphones 50 and 60 determined by the pulse 
adjustment devices 54 and 64 in Figure 5, are shared between these two devices by 
a data bus. 

An important feature of any bilateral fitting is that the frequency to electrode 
allocation in each ear should be arranged so that particular frequency bands are 
allocated to pitch matched electrodes in each ear. In the example shown in Figure 
4, each of the microphones 50 and 60 are arranged on opposite sides of the head, 
and separate processing of the signals for each microphone is provided. In this 
case, the pulse adjustment devices 54 and 64 each act to determine a matching 
template passing the maximum power compared to the remaining templates. Each 
of the pulse adjustment devices 54 and 64 then determines the power passed by the 
corresponding template in the other auditory prosthesis device. 

If the power passed by this corresponding template is less than the power 
passed by the matching template with which each pulse adjusting device is 
associated, the stimulation pulse adjustment device will adjust the simulation pulses 
as described hereabove. In this way, modification of the stimulation pulses need 
only occur on the side determined to have the larger output signal, this being the 
side where the dominant sound source is located. The template passing the 
maximum power in each auditory prosthesis device is used to "mark" the electrodes 
corresponding to the features on that side (i.e. with fundamental frequency 
modulations or temporal offsets) or alternatively to create fundamental frequency 
based control intervals, provided the power passed by the corresponding template in 
the other auditory prosthesis device is less than the power passed by that matching 
template. 

In the case of two concurrent complex sounds being present, if they are 
spatially separated, separate modifications to the stimulation pulses on each side can 
be made. For example, when in the presence of two complex acoustic sounds, the 
pulse adjustment device 54 determines that there are two matching templates 
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passing power maxima compared to the remaining templates in the bank 52 of 
templates, the pulse adjustment device 54 can adjust the stimulation pulses 
generated by the pulse generator 53 in accordance with the estimated fundamental 
frequency corresponding to the first power maximum, whilst the pulse adjustment 
device 64 adjusts the stimulation pulses generated by the pulse generator 63 in 
accordance with the estimated fundamental frequency corresponding to the second 
power maximum. 

In addition, or alternatively to the above bilateral scheme, stimulation pulses 
applied to electrodes corresponding to one or more formants of a complex acoustic 
sound by one of the auditory prosthesis devices can be de-correlated with 
stimulation pulses applied by the other of the auditory prosthesis devices to the 
corresponding electrodes in that other device. In other words, when the pulse 
adjustment device 54 in the example shown in Figure 5, adjusts pulses applied to 
one or more electrodes which correspond to one or more formants of a complex 
acoustic sound, the pulse adjustment device 64 acts to apply stimulation pulses that 
are de-correlated with the stimulation pulses adjusted by the pulse adjustment 
device 54, to one or more electrodes in the other auditory prosthesis device which 
correspond to the same one or more formants. The de-correlation may be effected 
by applying a different electrode stimulation rate to electrodes corresponding to the 
same formants in each auditory prosthesis device. This can be achieved in practice 
by incrementing the position of these electrodes in the order of electrode activation 
in the temporal pattern of pulses applied to the electrode array each electric 
stimulation period. 

It is to be understood that various modifications and/or additions may be 
made to the above described auditory prosthesis device and method for operating 
the auditory prosthesis device without departing from the spirit or ambit of the 
present invention. 

Dated: 28 February, 2003 
FREEHILLS CARTER SMITH BEADLE 
Patent Attorneys for the Applicant 
THE UNIVERSITY OF MELBOURNE 
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